Generalization error of three layered learning model in bayesian estimation

نویسندگان

  • Miki Aoyagi
  • Sumio Watanabe
چکیده

In this paper, we obtain the asymptotic forms of the generalization errors for some three layered learning models in Bayesian estimation. The generalization error measures how precisely learning models can approximate true density functions which produce learning data. We use a recursive blowing up process for analyzing the Kullback function of the learning model. Then, we have the maximum pole of its zeta function which is defined by the integral of the Kullback function and an a priori probability density function. In [1, 2], it was proved that the maximum pole of the zeta function asymptotically gives the generalization error of the hierarchical learning model.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Learning efficiency of redundant neural networks in Bayesian estimation

This paper proves that the Bayesian stochastic complexity of a layered neural network is asymptotically smaller than that of a regular statistical model if it contains the true distribution. We consider a case when a three-layer perceptron with M input units, H hidden units and N output units is trained to estimate the true distribution represented by the model with H(0) hidden units and prove ...

متن کامل

Algebraic geometrical methods for hierarchical learning machines

Hierarchical learning machines such as layered perceptrons, radial basis functions, Gaussian mixtures are non-identifiable learning machines, whose Fisher information matrices are not positive definite. This fact shows that conventional statistical asymptotic theory cannot be applied to neural network learning theory, for example either the Bayesian a posteriori probability distribution does no...

متن کامل

Resolution of Singularities and the Generalization Error with Bayesian Estimation for Layered Neural Network

Hierarchical learning models such as layered neural networks have singular Fisher metrics, since their parameters are not identifiable. These are called non-regular learning models. The stochastic complexities of non-regular learning models in Bayesian estimation are asymptotically obtained by using poles of their zeta functions which are the integrals of their Kullback distances and their prio...

متن کامل

A Surface Water Evaporation Estimation Model Using Bayesian Belief Networks with an Application to the Persian Gulf

Evaporation phenomena is a effective climate component on water resources management and has special importance in agriculture. In this paper, Bayesian belief networks (BBNs) as a non-linear modeling technique provide an evaporation estimation  method under uncertainty. As a case study, we estimated the surface water evaporation of the Persian Gulf and worked with a dataset of observations ...

متن کامل

A Surface Water Evaporation Estimation Model Using Bayesian Belief Networks with an Application to the Persian Gulf

Evaporation phenomena is a effective climate component on water resources management and has special importance in agriculture. In this paper, Bayesian belief networks (BBNs) as a non-linear modeling technique provide an evaporation estimation  method under uncertainty. As a case study, we estimated the surface water evaporation of the Persian Gulf and worked with a dataset of observations ...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2006